Japanese Dialogue Corpus of Multi-Level Annotation

This paper describes a Japanese dialogue corpus annotated with multi-level information built by the Japanese Discourse Research Initiative, Japanese Society for Artificial Intelligence. The annotation information consists of speech, transcription delimited by slash units, prosodic, part of speech, dialogue acts and dialogue segmentation. In the project, we used the corpus for obtaining new findings by examining the relationship between linguistic information and dialogue acts, that between prosodic information and dialogue segment, and the characteristics of agreement/disagreement expressions and non-sentence elements. 1 I n t r o d u c t i o n This paper describes a Japanese dialogue corpus annotated with multi-level information such as speech, linguistic and discourse information built by the Japanese Discourse Research Initiative, supported by Japanese Society for Artificial Intelligence. Dialogue corpora are now indispensable to speech and language research communities. • The corpora have been used not only for examining the relationship between speech and linguistic phenomena, but also for building • speech and language understanding systems. Sharing corpora among researchers is most desirable since creating the corpora needs considerable cost like writing and revising annotation manuals, annotating the data, and checking the consistency and reliability of the annotated data. Discourse Research Initiative was set up in March of 1996 by US, European, and Japanese researchers to develop standardized discourse annotation schemes (Carletta et al., 1997; Core et al., 1998). The efforts of the initiative have been called 'standardization', but this naming is misleading at least. In typical standardizing efforts, as done in audio-visual and telecommunication technologies, commercial companies try to expand the market for their products or interfaces by the standard. The objective of standardizing efforts in discourse is to promote interactions among discourse researchers and thereby provide a solid foundation for corpus-based discourse research, dispensing with duplicating resource making efforts and increasing sharable resources. In cooperation with this initiative, Japanese Discourse Research Initiative has started in Japan in May 1996, supported by Japanese Society for Artificial Intelligence (JDRI, 1996; Ichikawa et al., 1999). The activities of the initiative involve: creating and revising annotation schemes based on the survey of the existing schemes and annotation experiments, annotating corpora based on the proposed annotation schemes, and doing research using the corpora not only for examining the utility of the schemes and corpora but also for obtaining new findings.